What We Do and Don't Know about Software Development Effort Estimation

OVERWHELMING EVIDENCE DOCUMENTS a tendency toward cost and effort overruns in software projects. On average, this overrun seems to be around 30 percent.1 Furthermore, comparing the estimation accuracy of the 1980s with that reported in more recent surveys suggests that the estimation accuracy hasn’t changed much since then. (The only analyses that suggest a strong improvement in estimation accuracy are those by the Standish Group. The extreme improvement in estimation accuracy over the years suggested by its Chaos Reports is, however, probably just a result of an improvement in its own analysis methods from a selection overrepresented by problem projects to a more representative selection.2) Estimation methods haven’t changed much either. In spite of an extensive research on formal estimation models, the dominating estimation method is still expert estimation.3 An apparent lack of improvement in estimation accuracy doesn’t mean that we don’t know more about effort estimation than before. In this article, I try to summarize some of the knowledge I believe we’ve gained. Some of this knowledge has the potential of improving the estimation accuracy, some is about what most likely will not lead to improvements, and some is about what we know we don’t know about effort estimation. The full set of empirical evidence I use to document the claims I make in this summary appear elsewhere.1